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Roadmap  Intent 


Help  evaluate  the  ULS  systems  relevance  of  existing  or  planned  research 

•  The  roadmap  structure  explicitly  shows  a  ULS  system  perspective 
Prioritize  research  funding 

•  The  roadmap  provides  a  basis  for  determining  which  research  is  most 
critical/relevant/impactful  for  achieving  a  future  ULS  systems  capability 

Framework  for  incorporating  additional  ULS  systems  research 

Motivate  Research 

•  The  roadmap  shows  how  an  individual  research  initiative  supports  one  or 
more  ULS-system  technical  challenges 

Put  Research  in  Context 


— ULS  Systems  Research  Roadmap 

Software  Engineering  Institute  Carnegie  Mellon  March  2008 

r  ©  2007  Carnegie  Mellon  University 


The  Roadmap  Root:  A  Warfighter  Capability 


The  ULS  System  report  mentioned  six  capabilities  needed  by  the  DoD 

The  roadmap  combines  two  of  them  (C1/C6)  into  a  single  capability  to 
show  research  relevance  to  desired  military  capabilities 

•  Common  Relevant  Operational  Picture  (CROP):  Maintain  coherent 
common  operating  picture 

—  across  echelons,  services,  and  coalitions  in  a  mix  of  ultra-large-scale 
environments  (Cl) 

—  applying  local  context  to  global  information  sources  to  ensure  use  of  the 
right  data  any  time,  any  place,  for  any  mission  (C6) 
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A  Needed  Warfighter  Capability 


Common  Relevant  Operational  Picture:  Maintain  coherent  common  operating  picture  by 
rapidly  collecting,  processing,  disseminating,  and  protecting  information  spanning  echelons, 
services,  and  coalitions  across  a  mix  of  ultra-large-scale  environments.  Apply  local  context  to 
global  information  sources  to  ensure  use  of  the  right  data  any  time,  any  place,  for  any  mission. 
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Technical  Observations 
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Roadmap  Example 

Observation 


Different  users 
have  different  info 
needs  based  on 
their  role  and 
context 
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ULS  System  Perspectives 


Users  are 
developers,  i.e, 
they  can  augment 
the  system  to  fit 
their  needs 
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Previous  Roadmap  Example 


Different  users 
have  different  info 
needs  based  on 
their  role  and 
context 

I 

Users  are  developers, 

i.e,  they  can  augment  the 
system  to  fit  their  needs 


User  Customization 


Tech  challenge: 

Feed  user 

Provide  users  with  the 

needs  back  to  a 

ability  to  manipulate 

development 

and  customize  info 

group,  who 

contained  in  the 

creates/deploys 

CROP 

changes 

V 

Research  objective  2 

Support  user  content 
customizations  that  can  be 
disseminated  to  others  with 
similar  needs,  who  then 
develop  them  further 
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Augmented  Roadmap  Example 


CROP  capability  evolves 
non-uniformly  in  its 
structure,  components, 
and  uses 


Manage 
differences  rather 
than  eliminate 
them 


Different  users 
have  different  info 
needs  based  on 
their  role  and 
context 


Users  are  developers, 

i.e,  they  can  augment  the 
system  to  fit  their  needs 


User  Customization 


Tech  challenge: 

Feed  user 

Provide  users  with  the 

needs  back  to  a 

ability  to  manipulate 

development 

and  customize  info 

group,  who 

contained  in  the 

creates/deploys 

CROP 

changes 

Research  objective  2 

Support  user  content 
customizations  that  can  be 
disseminated  to  others  with 
similar  needs,  who  then 
develop  them  further 
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Research  objective  1 

Ensure  system  stability  and 
QoS  as  components  and 
usage  changes 


► 


6.5.3 

Evolutionary 

Configuration 

and 

Deployment 

Software  Engineering  Institute 


4 


6.6.4 

Enforcing 

Quality 

Attributes 


Research  Topic  from 
ULS  Systems  Report 


Carnegie  Mellon 
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6.1  Research 
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Research  objective  1 

Ensure  system  stability  and 
QoS  as  components  and 
usage  changes 


6.5.3 

Evolutionary 

Configuration 

and 

Deployment 


6.6.4 

Enforcing 

Quality 

Attributes 


6.5.S/6.6.4 

Investigate  certification  techniques  that  can 
ensure  adaptive  systems  only  operate  within 


6.S.3/6.6.4 

Develop  models/algorithms/tools  that  allow 
validating  key  functional  properties  before 
and  during  SW  updates 

6.5.3*76.5.4 

Determine  how  to  allow  user-defined 
capability  to  dynamically  interact  with 
existing  ontologies  and  user  interfaces 


6.5.3* 


Provide  ability  for  users  to  build  their  own, 
’  localized  ontologies  (perhaps  as  part  of 
Army’s  Development  Network) 


6.2/3  Research 


6.5.3* 


Implement  SOA  so  users  can  build  their  own 
services  and  so  service  priorities/performance 
can  be  tracked  across  the  enterprise 
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Research  objective  1 

Ensure  system  stability  and 
QoS  as  components  and 
usage  changes 


► 


6.5.3 

Evolutionary 

Configuration 

and 

Deployment 


4 


6.6.4 

Enforcing 

Quality 

Attributes 


6.5.3/6.6.4 

Investigate  certification  techniques  that  can 
ensure  adaptive  systems  only  operate  within 
safe,  correct,  and  stable  configurations 


6.5.3/6.6.4 

Develop  models/algorithms/tools  that  allow 
validating  key  functional  properties  before 
and  during  SW  updates 


6.5.3*76.5.4 

Determine  how  to  allow  user-defined 
capability  to  dynamically  interact  with 
existing  ontologies  and  user  interfaces 


Provide  ability  for  users  to  build  their  own, 
localized  ontologies  (perhaps  as  part  of 
Army’s  Development  Network) 


▼ 


6.6.4 


Develop  quality  attribute  enforcement 
protocols,  their  associated  quality 
attribute  theories,  and  the  means  for 
dynamic  (online)  adaptation  of  both 


6.5.3* 


Implement  SOA  so  users  can  build  their  own 
services  and  so  service  priorities/performance 
can  be  tracked  across  the  enterprise 
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Research  objective  1 

Ensure  system  stability  and 
QoS  as  components  and 
usage  changes 


Research  objective  2 

Support  user  content 
customizations  that  can  be 
disseminated  to  others  with 
similar  needs,  who  then 
develop  them  further 


6.5.3 

Evolutionary 

Configuration 

and 

Deployment 


6.6.4 

Enforcing 

Quality 

Attributes 


6.5.3/6.6.4 

Investigate  certification  techniques  that  can 
ensure  adaptive  systems  only  operate  within 
safe,  correct,  and  stable  configurations 


6.5.3/6.6.4 

Develop  models/algorithms/tools  that  allow 
validating  key  functional  properties  before 
and  during  SW  updates 


6.1.4 

Fostering  Non- 
Competitive 
Social 

Collaboration 


6.1.4/6.6.4 

Produce  a  variety  of  non-competitive 
development  models  relevant  to  the  types 
of  adaptations  needed  for  the  CROP  while 
ensuring  that  these  development  models 
can  achieve  the  level  of  needed  quality. 


▼ 


6.5.3*/6.5.4 

Determine  how  to  allow  user-defined 
capability  to  dynamically  interact  with 
existing  ontologies  and  user  interfaces 


Provide  ability  for  users  to  build  their  own, 

’  localized  ontologies  (perhaps  as  part  of 
Army’s  Development  Network) _ 

6.5.3* 

Implement  SOA  so  users  can  build  their  own 
'  services  and  so  service  priorities/performance 
can  be  tracked  across  the  enterprise 


6.6.4 

Develop  quality  attribute  enforcement 
protocols,  their  associated  quality 
attribute  theories,  and  the  means  for 
dynamic  (online)  adaptation  of  both 
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Research  objective  1 

Ensure  system  stability  and 
QoS  as  components  and 
usage  changes 


6.5.3 

Evolutionary 

Configuration 

and 

Deployment 


4 


6.6.4 

Enforcing 

Quality 

Attributes 


6.S.3/6.6.4 

Investigate  certification  techniques  that  can 
ensure  adaptive  systems  only  operate  within 
safe,  correct,  and  stable  configurations 


6.S.3/6.6.4 

Develop  models/algorithms/tools  that  allow 
validating  key  functional  properties  before 
and  during  SW  updates 


Research  objective  2 

Support  user  content 
customizations  that  can  be 
disseminated  to  others  with 
similar  needs,  who  then 
develop  them  further 


6.1.4 

Fostering  Non- 
Competitive 
Social 

Collaboration 


6.1.4/6.6.4 

Produce  a  variety  of  non-competitive 
development  models  relevant  to  the  types 
of  adaptations  needed  for  the  CROP  while 
ensuring  that  these  development  models 
can  achieve  the  level  of  needed  quality. 


6.5.3*/6.5.4 

Determine  how  to  allow  user-defined 
1  capability  to  dynamically  interact  with 
existing  ontologies  and  user  interfaces 

6.5.3* 

Provide  ability  for  users  to  build  their  own, 
’  localized  ontologies  (perhaps  as  part  of 
Army’s  Development  Network) 


6.6.4 


Develop  quality  attribute  enforcement 
protocols,  their  associated  quality 
attribute  theories,  and  the  means  for 
dynamic  (online)  adaptation  of  both 


6.5.3* 

Implement  SOA  so  users  can  build  their  own 
services  and  so  service  priorities/performance 
can  be  tracked  across  the  enterprise 
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Roadmap  Structure  and  Development  Process 


Start  with:  a  needed  ULS  system  capability 

Make:  Observations  about  this  capability 

•  Example:  user  needs  change  dynamically 

Use:  ULS  systems  perspective  (contrasted  with  conventional  approach) 
Identify:  fechnical  challenge  (related  to  ULS  systems  perspective) 

•  Contrast  with  the  “usual”  technical  challenge 
Restate  challenge  as:  Research  objective 
Cite:  ULS  Systems  report  Research  Topic 

Define  Research  Initiatives:  Several  supporting  each  research  objective 
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Incentivize  User 


CROP  capability  evolves 
non-uniformly  in  its 
structure,  components, 
and  uses 


Different  users 
have  different  info 
needs  based  on 
their  role  and 
context 


Manage 
differences  rather 
than  eliminate 
them 


Exploit  self- 
interest  rather 
than  overriding  it 


Tech  challenge:  Achieve 
globally  appropriate  user/ 
system  performance  while 
exploiting  competitive  self- 
interested  behavior 


Determine  optimal 
behavior  and  train 
people  to  behave 
optimally 


Research  objective  3 

Incentivize  globally 
appropriate  user  behavior 
by  providing  rules  that 
encourage  appropriate 
system  use 
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Automatic  System  Adaptation 


User  Relevance  and  Robustness 


User  needs  for  info 
change  dynamically 


System  connectivity 
and  info  flow 
changes  dynamically 


People  will  (mis)use 
the  system  in 
unexpected  ways, 
stressing  HW  and  SW 


Automated  adaptation  rather 
than  manual  intervention  to  deal 
with  failures  and  differing 
operational  conditions 


Failures  are  normal,  i.e. , 
because  of  scale,  even  unlikely 
events  will  occur 


User-Relevant  Info 


Tech  challenge: 

Dynamically  adjust  info  offered 
as  the  warfighter’s  tasks, 
context,  and  network 
connectivity  change  over  time. 


Predetermine  the  info 
to  be  transmitted 


Robust  System 


Tech  challenge: 

Design  systems  to  be  robust 
against  unlikely  usage  and 
operating  conditions. 


Focus  on 
functionality  and 
normal  usage 


Research  objective  5 

Anticipate  user  needs  in 
tactical  environments  with 
limited  resources,  provide 
appropriate  information  to 
those  users,  and  learn  needs 
of  varying  classes  of  users 


Research  objective  6 

Develop  design  and 
certification  methods  that 
increase  a  system’s 
robustness  against 
unexpected  usage  and 
operating  conditions 
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ULS  Systems  Research  Topics  In/ Not  In  Roadmap 


6. 1. 1  Context-Aware  Assistive  Computing 

6. 1.2  Understanding  Users  and  Their  Contexts 

6. 1.3  Modeling  Users  and  User  Communities 

6. 1.4  Fostering  Non-Competitive  Social 
Collaboration 

6.1.5  Longevity 

6.2. 1  Algorithmic  Mechanism  Design 

6.2.2  Metaheuristics  in  Software  Engineering 

6.2.3  Digital  Evolution 

6.3.1  Design  of  All  Levels 

6.3.2  Design  Spaces  and  Design  Rules 

6.3.3  Harnessing  Economics  to  Promote  Good 
Design 

6.3.4  Design  Representation/Analysis 

6.3.5  Assimilation 

6.3.6  Determining  and  Managing  Requirements 

6.4.1  Expressive  Representation  Languages 


6.4.2  Scaled-Up  Specification,  Verification,  and 
Certification 

6.4.3  Computational  Engineering  for  Analysis  and 
Design 

6.5.1  Decentralized  Production  Management 

6.5.2  View-Based  Evolution 

6.5.3  Evolutionary  Configuration  and  Deployment 

6.5.4  in  Situ  Control  and  Adaptation 

6.6.1  Robustness,  Adaptation,  and  Quality  Attributes 

6.6.2  Scale  and  Composition  of  Quality  Attributes 

6.6.3  Understanding  People-Centric  Quai.  Attr. 

6.6.4  Enforcing  Quality  Requirements 

6.6.5  Security,  Trust,  and  Resiliency 

6.6.6  Engineering  Management  at  Ultra-Large 
Scales 

6.7.1  Policy  Definition  for  ULS  Systems 

6.7.2  Fast  Acquisition  for  ULS  Systems 

6.7.3  Management  of  ULS  Systems 
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Roadmap  Intent 


Help  evaluate  the  ULS  systems  relevance  of  existing  or  planned  research 

•  The  roadmap  structure  explicitly  shows  a  ULS  system  perspective 
Prioritize  research  funding 

•  The  roadmap  provides  a  basis  for  determining  which  research  is  most 
critical/relevant/impactful  for  achieving  a  future  ULS  systems  capability 

Framework  for  incorporating  additional  ULS  systems  research 

Motivate  Research 

•  The  roadmap  shows  how  an  individual  research  initiative  supports  one  or 
more  ULS-system  technical  challenges 

Put  Research  in  Context 
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